Fixation, transient landscape and diffusion's dilemma in 
stochastic evolutionary game dynamics 

Da Zhou I'fl 

-^School of Mathematical Sciences, Peking University, Beijing 100871, China and 
"^MOE Key Laboratory of Bioinformatics and Bioinformatics Division, 
TNLIST/ Department of Automation, 
^ ■ Tsinghua University, Beijing 100864, China 



Hong Qian ^ 

0^ ■ ^Department of Applied Mathematics, 



pq , University of Washington Seattle, WA 98195, USA 

CM : 

Q ■ Abstract 
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, Agent-based stochastic models for finite populations have recently received much attention in 



(N 



the game theory of evolutionary dynamics. Both the ultimate fixation and the pre-fixation tran- 



CN ■ sient behavior are important to a full understanding of the dynamics. In this paper, we study the 

> ; 

*0 . transient dynamics of the well-mixed Moran process through constructing a landscape function. It 

'NT 



is shown that the landscape playing a central theoretical "device" that integrates several lines of 
inquiries: the stable behavior of the replicator dynamics, the long-time fixation, and continuous 
diffusion approximation associated with asymptotically large population. Several issues relating to 
the transient dynamics are discussed: 1) Multiple time scales phenomenon associated with intra- 
^ \ and inter-attractoral dynamics; 2) discontinuous transition in stochastically stationary process akin 



to Maxwell construction in equilibrium statistical physics; and 3) the dilemma diffusion approx- 
imation facing as a continuous approximation of the discrete evolutionary dynamics. It is found 
that rare events with exponentially small probabilities, corresponding to the uphill movements and 
barrier crossing in the landscape with multiple wells that are made possible by strong nonlinear 
dynamics, plays an important role in understanding the origin of the complexity in evolutionary, 
nonlinear biological systems. 
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I. INTRODUCTION 



One of the salient features of the stochastic evolutionary game dynamics for finite pop- 
ulations is the fixation That is, no matter how the initial strategies are distributed 
in a population, the system will eventually be fixated to only one strategy. In general, this 
phenomenon can be theoretically explained in terms of a Markov process with absorbing 
state(s) {4, 5|: The limiting theory of Markov processes tells us that the finite-state Markov 
chains with absorbing states will eventually be trapped into one of the absorbing states 
as time passes. The well-known Wright-Fisher model , Moran process js], ^ and the 



pairwise comparison processes 
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11| all belong to this class. A further classification of Dar- 



winian selection scenarios based on the fixation descriptions has already been established in 
stochastic game dynamics 12Nl7l| . Another important class consists of evolutionary dynam- 
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20|. 



ics with mutations where an ergodic mutation- select ion equilibrium can be reached 
The latter is out of the scope of our study; However, the landscape introduced in the present 
work provides a unified perspective for both classes of processes. 

In addition to the ultimate fixation, attention should also be paid to the time-dependent, 
pre-fixation transient behavior for several reasons 



21 



231 ]: On one hand, the process to 



fixation is intimately dependent upon both the transient movements before absorption and 
the one last step to fixation. That is, studying the transient dynamics provides important 
insights into the final fixation behavior. On the other hand, sometimes the time for a true 



fixation is too long to be observed 13| and the relevant time scale can be shorter 24 



26|. 



In this case, the transient dynamics provides a more appropriate description. Furthermore, 
examinations of transients can yield a mechanistic understanding of the persistence and 



coexistence in complex biological dynamics, especially in ecosystems [27[. Indeed, it has 
been found that the pre-fixation transient dynamics could be an essential explanatory aspect 



32|. 



of characterizing the stochastic fluctuations raised from finite populations [28|- 

The theory of quasi- st at ionarity is a widely applied, standard technique of studying the 
pre-fixation process 33[. It defines the subchain with the absorbing states removed. Based 
on this approach, we present an extended analysis for the transient dynamics of the well- 
mixed frequency-dependent Moran process. An ergodic conditional stationary distribution 
is used to characterize the pre-fixation process. As a result of the law of large numbers, this 
stationary distribution approaches to a singular distribution in the infinite population limit. 
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The corresponding large deviation rate function [3J-|36j, which is population-size indepen- 
dent, is shown to be a landscape. This transient landscape has a Lyapunov property with 
respect to the corresponding deterministic replicator dynamics, providing a potential-like 
function for visualizing the transient stochastic dynamics. Ideas related to the transient 
landscape of Moran process have been discussed in the past: Claussen and Traulsen 29 1 
studied non-Gaussian stochastic fluctuations based on the conditional stationary distribu- 
tion. It is also a general feeling that one can use the negative logarithm of the stationary, 
or conditional stationary distribution as the potential in evolutionary dynamics, following 
an analogue to Boltzmann's law in statistical mechanics. However, it is important to point 
out that a stationary distribution usually collapses to singular supports in the infinite pop- 
ulation limit, while our large deviation rate function iplx) is supported on the whole space 
and it is independent of system's size. Therefore, in terms of the analogue to Boltzmann's 
law, we are effectively identifying the system's size as the inversed temperature which tends 
to infinity for a deterministic limit. 

Even though our analysis is based on the one- dimensional Moran process, this idea is 
general. It can be applied to many other multi-dimensional evolutionary game dynamics 
with finite populations, with or without detailed balance b7|. For the latter case, the land- 

n n 

scape itself is an emergent property of the dynamics. With respect to Moran process, P, 113[ 
also discovered the expression of tp{x) from a different origin, via their approximated calcu- 
lation of fixation probability for large population size. We shall show that this connection 
is a nice mathematical property of the ip{x) function for the processes in one-dimensional 
case, but its generalization to multi-dimensional cases is not obvious. More specifically, 
for multi-dimensional systems with multiple alleles, the fixation probability does not nat- 
urally give a landscape. The large deviation rate function, however, can be generalized to 
multi-dimensional Markov processes, as indicated by the Freidlin-Wentzell theory 3J|. The 
landscape we introduced is also consistent with the landscape theory for other population 
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dynamics, e.g., chemical, that is ergodic without fixation [37, 

There are two fundamentally different types of movements in this landscape that require 
separated attention, (i) "Downhill movements" which have deterministic counterparts: The 
local minima {transient attractors) in this landscape correspond to the stable points in the 



replicator dynamics 



39j . That is, these transient attractors are in direct agreement with the 



evolutionarily stable strategies (ESSs) 40|, |4l||. (ii) "Uphill movements" which are rare and 
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without a deterministic correspondence. In general, rare events take exponentially long time; 
one needs to take multiple time scales into consideration in understanding the appropriate 
fluctuation descriptions for the transient dynamics as well as eventual fixation. This is 
particularly relevant in the anti-coordination games. 

Furthermore, the concept of stochastic bistability is studied in the coordination games. In 
this case, the downhill and uphill movements in the landscape dominate "intra-attractoral" 



and "inter-attractoral" dynamics respectively [37(|. It is shown that a Maxwell-type con- 
struction from classic phase transition theory in statistical physics 42| is necessary as the 
population size tends to infinity, i.e., only one of attractors should be singled out in such a 
construction — It corresponds to the global minimum of the landscape. This is not present 
in the bistable deterministic dynamics; it raises the novel issue of ultimate fixation. It did 
not escape our notice that it is the exponentially long-time search that ultimately finds the 
global minimum in a "non-convex optimization" 43|. 

Another important issue directly relating to the transient dynamics is the diffusion ap- 
proximation 4J-|46[ . With the conventional truncation of Kramers- Moyal (KM) expansion, 
the discrete stochastic Moran process for large populations has been approximated by a 



stochastic differential equation 
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481], with absorbing Dirichlet boundary conditions. If 



one replaces the absorbing boundary conditions with the reflecting ones, we can also derive 
an ergodic stationary distribution from the Fokker-Planck equation of this diffusion process. 
It will be shown that this stationary distribution is in fact the "conditional" stationary 
distribution for the process with absorbing boundary conditions. In a comparison of the 
transient dynamics between the original Moran process and its continuous counterpart, it 
is shown that even though the KM diffusion is valid in finite time as a local dynamical 
approximation, it could lead to incorrect approximation in global inter-attractoral dynam- 
ics. In bistable game systems, particularly, the KM diffusion could single out a different 
stable point from that of the original process for large but finite populations. Moreover, 
enlightened by Hanggi et al.'s work 49[, we also consider their diffusion approximation that 
provides the correct global dynamics. However, this diffusion process gives incorrect finite 
time stochastic dynamics. 

Now we have a diffusion's dilemma: The truncated KM diffusion gives the correct finite 
time stochastic dynamics as the original Moran process with large population size (this is 
guaranteed both by the so called van Kampen's system size expansion 50| and Kurtz's 
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theorem 
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52|), but wrong stationary distribution. On the other hand, Hanggi et aL's 



diffusion, which is unique in providing the correct stationary distribution as well as deter- 
ministic limit, is wrong for the finite time stochastic dynamics. To further illustrate this 
diffusion's dilemma, a simple example is present. By investigating the first passage times, it 
is found that the failure of exponential approximation in the uphill movement could be the 
origin of the difficulties of diffusion approximation. In other words, diffusion approximation 
is a second-order polynomial expansion for the Kolmogorov forward equation of the origi- 
nal discrete process, which can give the correct Gaussian dynamics near the stable point; 
However, the inter-attractoral global dynamics, determined by the barrier crossing events 
with exponential small probabilities, should be approximated in the level of exponential 
asymptotics. 

This paper is organized as follows: In Sec. II, we introduce the frequency-dependent 
Moran process. Then we give the transient description of the Moran process in Sec. Ill, 
where the transient landscape ip{x) is constructed. It is shown that this landscape as a "glue" 
holds the deterministic replicator dynamics, the fixation and the problem of Maxwell-type 
construction together. Diffusion's dilemma is discussed in Sec. IV. The discussions are 
included in the last section. 



II. FREQUENCY-DEPENDENT MORAN PROCESS 



To study evolutionary game theory in finite populations, Nowak et al. |9| generalized 



Moran's classical population genetic model js] by using frequency-dependent fitness. Con- 
sider a population of N individuals playing a symmetric 2x2 game with strategies A and 
B, the payoff matrix is 

f a b\ 

where all the entries in the matrix are assumed to be non-negative. If i players follow 

strategy A, and N — i play B, the the average payoff of an individual of A is 

a(z-l) + &(iV-z) 
Fa = ^^31 ' (2) 

where self-interaction is excluded, and also for B is 



Fitness is assumed to be a linear combination of background fitness and the payoff as follows: 



fX = l-w + wFX, 



(4) 



fB = l-w + wF}„ (5) 

where w E [0, 1] denotes the intensity of selection. 

At each time step, one individual is chosen to reproduce proportional to its fitness, then 
the offspring replaces a randomly chosen individual, so that the population size is constant. 
The evolutionary mechanism in this Moran process, thus, can be conveniently described as 
a Markov chain. If we denote X as the the number of individuals following strategy A, 
then X is actually a finite-state birth-death process with discrete-time steps, which can be 
expressed as follows: 







1: 



--{N 



T+ 

^ N-2 



T+ 



where the birth and death probabilities conditional on the present state X = i are 



P{i^i + l) = 

= P{i^i-l) 
T? = Pii ^ { 



N 



^fX+(N-^)f|, N 

z/A+(iV-z)/ljiV' 
= 1-T+ -Tr. 



(6) 

(7) 
(8) 



So the Markov transition probability matrix of this process can be denoted as 

/ 1 ■■■ \ 

Tf 1 - (T+ + T{) T+ ■■■ 

Q 



(9) 



\ ■■■ 1 / 

It is easy to see that X = and are two absorbing boundaries for the system, so the 
limiting distribution of the Moran process can be denoted as 



(1 - pi)6o + pidN, 



(10) 



where pi is the fixation probability that i individuals of strategy A succeed in taking over 
the entire population, and 6o (or 6n) is a Dirac mass at (or A^). In other words, except 
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for the two pure-strategic absorbing states, all the mixed states are transient. This process 
is different from an irreducible Markov chain whose ergodic behavior is supported on the 
whole state space. Thus in the Moran process, it is natural to investigate the conditions for 
selection to favor successful fixation by calculating the corresponding fixation probabilities 



12|. 



III. TRANSIENT ANALYSIS 

The fixation is intimately dependent upon the transient behavior before absorption. To 
show this, we shall divide the process to fixation into two parts: One is the irreducible 
transitions among the transient states before absorption, the other is the last step to fixation. 
This distinction has a very clear evolutionary meaning: the former is a consequence of "global 
dynamics" while the latter is a rather local event. It is also worth mentioning that these 
two scenarios correspond so-called "golf-course search" and "funnel perspective" in the field 



of protein folding [53|. By explicitly studying these two different problems, one is able to 
address the "dynamic nature of fixation" : Is the dynamics toward fixation inherent in the 
evolutionary dynamics? The investigation of the transients provides another perspective to 
understand the fixation. Further, in order to describe the inherent fluctuations arising from 



finite popu 



aspect 
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ations, the transient dynamics has been proved to be an essential explanatory 



3Q|. 



A. Conditional stationary distribution and the transient landscape 

A natural mathematical idea to describe the transient behavior is to concentrate on the 



distribution conditional on the subspace of the mix-strategic states [33|. We consider an 
auxiliary process X*{t) close to the original Moran process X{t). X*{t) can be described 
as the original process with the absorbing states removed, while all other birth and death 



probabilities remain unchanged |33|, [sj]. So the process of X*(t) can be expressed as 



-^l -'jV-2 

1^2^ - ■ -^{N -2)^{N -1). 
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Note that X*{t) is an ergodic Markov chain with a unique stationary distribution {rji}, 
yielding the following iterative relation 



Vi 



k=2 



fc-i 



T,: 



2,...,N 



1. 



(11) 



The interpretation of rj for the transient description of the original process X{t) is from 
the theory of quasi-stationarity 26|. It is shown that t] characterizes the occupation time 
distribution of the process before absorption, i.e. rji is the average times of visiting to i 
divided by the mean absorbing time (see appendix A). So we call t] the conditional stationary 
distribution of the original process. 

We write r] as r/^^-* for its dependence on the population size N, 



n. 



[N) 



(N) 



By expanding in the inverse of large population size A^, r/^^-* can be expressed as 

y4exp 



N 



X 



I 

N' 



where the leading item ip(x) is the large deviation rate function of r]^^^ 
Euler-MacLaurin summation formula, we have 
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55|. By the 



ilj{x) = — 



/ In 




Jo 


[T-{y)\ 



dy, 



where 



T-{y) 



x{l — x){l — w + w{ax + b{l — x))) 



x{l — w + w{ax + b{l — x))) + (1 — x)(l — w + w{cx + d{l — x))) 
x(l — x){l — w + w{cx + d{l — x))) 



x(l 



w 



(12) 

(13) 
(14) 



w{ax + 6(1 — x))) + {1 — x){l — w + w{cx + d{l — x))) 
We term ip{x) as the transient landscape, which will be shown to be of great importance 

in characterizing both deterministic and stochastic dynamics. It should be mentioned that 

our definition of the transient landscape can be extended to more general dynamics with 

multiple populations (see appendix B). An relation between iplx) and fixation probabilities 

in y,!!^ will be given in Eq. ( !T6|) (Also see appendix D). 



For the replicator equation 
dx(t) 



dt 



x{l — x)[{{a — b)x + &) — ((c — d)x + d)], 



It is easy to check that 

• , , , , dip(x) dx 



= - In 
< 0. 



1 — w + w{{a — h)x + h) 
1 — w + w((c — d)x + d) 



x{l — x)[((a — h)x + h) — ((c — d)x + 



So ip{x) has the Lyapunov property [56|, |57[, i.e. the dynamical behavior of the rephcator 
dynamics can be predicted by this transient landscape. Note that by our definition of 
ip{x), the Lyapunov property is nearly transparent, even for multi-dimensional systems (see 
appendix B). 

We classify the transient landscape into three main generic cases (see Fig. 1): 

1) Uni-well: a < c and b > d. iIj{x) decreases on [0,x*] and increases on Here 
X* = {d — b)/ {a — b — c + d) is the only minimal extreme point. 

2) Uni-barrier: a > c and b < d. ip{x) increases on [0,x*] and decreases on [x*, 1]. In 
this case, the landscape has two local minimal points at both x = and 1, separated by the 
peak at x*. 

3) Uphill/Downhill: a < c and b < d (or a > c and b > d). In this case, ip{x) increases 
(or decreases) on the whole interval [0, 1]. So x = 1 (or x=0) is the minimal point. 

Another seemingly trivial case is when a = c and b = d, which is of limited interest in the 
deterministic dynamics. However, this neutral case with flat landscape becomes important 
in the context of stochastic dynamics. An interesting result will be obtained for this case in 
connection to diffusion approximation (Sec. IIVBI) . 

Note that the rescaled conditional stationary distribution ri{x) can be expressed as 

ri{x) oc exp{—Nip{x)}, 

the landscape iIj{x) visualizes the transient dynamics: The transient system should spend a 
majority of time around local minimal point (s) in the landscape. So we term the minimal 
point(s) as the transient attractor(s). In the literature of physics, the transient attractor(s) 



show the properties of metastablity 58l-l60| . That is, although the "downhill movement" 
towards the local minimum in the landscape maintains the stability of the attractor, the 
"uphill movement" of crossing the barrier will drive the system to move from the local 
attractor to another on a larger time scale. With this observation, we will discuss the 
fixation from the viewpoint of the transient landscape. 



B. The transient landscape and fixation 



It is known that the process to fixation is intimately dependent upon both the transient 
movements before absorption and the one last step to fixation. Thus we have two cases: First, 
the fixation is an inherent result derived directly from the transient process. Second, the 
fixation shows distinctly different behavior from the transient process, i.e., the final fixation 
does not end up with attractive absorbing, but one last "unnatural" step to fixation. 

Based on the different generic cases of the transient landscape, it is found that in the uphill 
or downhill case, one of the two absorbing states is located at the transient attractor, we 
term this kind of absorbing state as the attractive absorbing state; the other absorbing state 
is called as the rare absorbing state. This classification of the absorbing states is directly 
linked to the work by Antal et al. We now denote the probability of fixation at A^, 

before reaching and starting from the initial state i, by pj. Similarly, fixation probability 
at starting from i is denoted by 7^. The explicit expression of pi, for example, can be 
derived from the following difference equation 2|: 



Pj = Tj Pi-i + (1 



with two boundary conditions 



Then we have 



Po = 0, pn = 1- 



1 + n~=\ uu 



pi 



where Xj = /T^ . When is sufficiently large |3|, [l3[, 

_ 1 +Ei^n -=17-/1? 



(15) 



1 + Ylk=i e^'^^*^/^) 
l + N j^e^^'^y^dy 
1 + N J^e^^iy)dy 

J^e^^(y)dy 



where x = i/N. Therefore, 



lim ^In-^p^^ 



lim — In — 7Ard-ri 



ip{x). 



(16) 
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Eq. ( !T6|) establishes a connection between our the transient landscape with fixation proba- 



bility. As pointed out by 



13] . in the downhill case that a > c and b > d, 



where 

^ |'cN(c/(c-d)) 
\ ^ "Adjl ^ 1 

5 (|)(a/{a-6))) ^ 

This result shows that the uphill fixation from — 1 to is a rare event with exponentially 
small probability, while the downhill fixation from to — 1 is a rather easy trip. This 
corresponds to our "rare" or "attractive" definition of the absorbing states. Similarly, in the 
uni-harrier case, both x = and 1 are the attractive absorbing states, whereas the barrier 
crossing probability from each side to another is exponentially small. 

In the uni-well case, however, the only transient attractor is located at the mixed state 
X*. In this case, the fixation is not an immediate result of the transient attraction. Antal 
et al. 



13| shows that the fixation time in this case is exponentially large with population 
size A^; while in the other two cases the fixation times have the same approximated order 
A^ In A^. This result is also completely in line with our classification of the fixation. 

The mismatch between the mixed transient attractor and the final absorbing fixation 
leads to multiple time scales issue in the process of evolution. Comparative studies of the 
mean first passage time to the attractor x* and the fixation time have been carried out in 
32|, |40|, showing the separation of the transient attractive time scale and the fixation time 



scale. 

Multiple time scales issue is of great importance in understanding evolutionary systems 



24 



27 1, esp ecially in explaining the coexistence and extinction of species in ecological systems 



25|. It has been reported that the relevant time scale to explain the coexistence of 



species in plankton [2J] is found in the short term (within a single season in their models). 
The time until species being extinct can be much longer than a single season. Accordingly, 
the coexistence can be explained here as a transient phenomenon. The mixed transient 
attractor, as the stable equilibrium in the transient dynamics, should be more relevant 
within a reasonable time scale. To realize the final fixation, the system has to escape from 
the attractor through going uphill on the landscape, collecting many unfavorable moves 
consecutively, for an extremely long time. 

11 



C. Stochastic bistability and Maxwell-type construction 



Bistabilit y (o r multistability) is one of the most interesting phenomena in the nonhnear 



systems 



45 



57 



6l| . For example, consider the rephcator dynamics 
dx{t) 



dt 



x{l - x)[{{a - b)x + b) - ((c - d)x + d)], 



bistabihty arises when a > c and b < d. In this case, x = and 1 are both stable, separated 
by the unstable fixed point x* = {d — b)/{a — b — c + d). Therefore, the characterizations 
of the bistability in the deterministic nonlinear systems should contain two things: One is 
where the attractors are, the other is the basins of attraction. 

One major problem in evolutionary game theory is the selection of multiple evolutionary 
stable strategies 2]J]. In the bistability case of the deterministic dynamics, the limiting 
behavior is determined by its initial state. That is, the measurement of the stability is 
closely dependent on the basins of attraction. The stable point with the larger basin of 
attraction can be seen as the risk-dominant strategy. 

In the context of stochastic evolutionary game dynamics, we can also discuss the noise- 



induced bistable phenomenon 



61 



62|. The bistability in the replicator dynamics corresponds 



to the uni-barrier case in the transient landscape, where both x = and 1 are the local 
minimal points in this landscape, separated by the barrier x* . We term this case with two 
transient attractors as the stochastic bistability. Furthermore, not only does the landscape 
cover the characterizations of the bistability in the replicator dynamics, but we can also give 
a straightforward comparison to these two stable states based on this landscape. From Eq. 

m 

Z"" ^^ -w + w{{a-b)y + b)^ , 

dy, 



ip{x) 



In 



so 



and 



1 — w + w{{c — d)y + d) 
m = 0, 



^(1) 



In 



1 — w + w{{a — b)y + b) 



dy 



- w + w{{c — d)y + d) 

I \n{l — w + w{{c — d)y + d))dy — / \n{l — w + w {{a — b)y + b))dy. 
Jo Jo 
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Without loss of generality, we set w = 1, then 



In [ay + h{l - y)] dy = In [cy + d{l - y)] dy 
Jo 
6 In 6 — a In a d In d — c In c 



a 



d 



We term this condition as the Maxwell-type construction 42|. Note that 

ri{x) oc exp{—Nip{x)}, 



so 



77(1) 



exp{iV(^(l)-^(0))}. 



When ip{0) > ^{1) 
when 'ip{0) < ip{l), 



rm 

r/(l) 



as N ^ oo. 



— 7- oo as N ^ oo. 



Thereby, even a slight difference between x = and 1 in the transient landscape can leads 
to a extreme disparity in the distribution (see Fig. 2). It is observed that except for the 
critical case, the system will select only one attractor, the global one, as the unique stable 
state with the increase of the population size 18|, |42|. In other words, the Maxwell- type 



construction always singles out the global minimum in the system, providing another useful 
criterion for the equilibrium selection. 



IV. DIFFUSION'S DILEMMA OF MORAN PROCESS 

Discrete Markov chain treatment of biological population systems is necessary for rel- 
atively small populations. For large populations it is convenient and desirable to apply 



a continuous approximation 



63 



64j . Beyond the replicator deterministic dynamics as a 



continuous limit, a diffusion-type process has long been much sought after. However, an 
important problem arising is the relation between the original discrete Markov chain and 
its approximated representation in term of a diffusion process 65| . The perspective of mul- 
tiple time-scale dynamics in the previous section provides a better understanding of this 
important problem. 

The insights we gained from the transient descriptions leads naturally to a comparative 
study of the original discrete-state Moran process and its continuous-path counterpart. 
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A. The Kramers-Moyal expansion and landscape via diffusion approximation 



It is known from the Kramers-Moyal diffusion theory in physics [45|] that the Moran 
process for large population size can be approximated by a stochastic differential equation. 
Moran process is a discrete-time, discrete-state Markov process; its Kolmogorov forward 
equation (sometimes called Master equation) has the form: 

P,+i(z) - P,(z) = P,i^ - l)Tt, + m + l)r,;i - P,{i)T- - mT^. (17) 

When is large, we take the scalings x = i/N, t' = t/N, and the probability density is 



f{x,t) = NPt{i) (we still write t' as t). By performing the truncated Kramers-Moya^ 
expansion of Eq. f|T71) . we have the following approximated Fokker-Planck equation 
df{x,t) d 1 fT+{x)+T-{x 



(KM) 



dt dx 

This corresponds to the stochastic differential equation 



((^^M-^-W)/)^^^(^;^^^/). (18) 



dx = (r+(x) - T~{x))dt + ^I^i^^l^^LJ^dBt. (19) 

where Bt is a Brownian Motion. In the limit of infinite population size, it is easy to see that 
Eq. fnUl) becomes the deterministic replicator dynamics 

X = T+{x) - T-{x). (20) 

In this way, this diffusion approximation links the stochastic Moran process and the macro- 
scopic nonlinear equation. 

We should note that the above truncated KM expansion is performed by taking the same 
scaling step of time and space with However, when a = c and b = d, i.e. the neutrality 
case, the transient landscape is flat with 

T+{x) = T-{x) 

for any x G (0, 1). In this case, the above scaling step is not valid any more. As a modiflcation, 
we take x = i/ \/N, then by performing the truncated KM expansion, we will have 

df{x,t) _ [ T+{x)+T-{x) ;\ 

dt ~ dx^\ 2 ^^-^^ 

This can be well explained by van Kampen's size expansion (see the Appendix C), which 
indicates that the scaling of the deterministic drift part should be different from that of the 

14 



fluctuated diffusion part — a well-known fact for the Law of Large Numbers and the Central 
Limit Theorem. 

We now consider the stationary distribution of the diffusion process in Eq. f llSp . The 
equation satisfied by the stationary distribution should be 



d 1 f T+{x)+T-ix) . 



If the boundary conditions are reflecting, the stationary distribution can be given by 



(22) 



7r(x) oc exp(— A^0(x)), 



(23) 



where 



(j){x) 



T+{y)-T-{y) 
LT+(y)+T-(|/), 



dy. 



However, for the Moran process with absorbing boundaries Tq = and 



approximation should have corresponding absorbing boundaries 



45 



6|: 



N 



(24) 

0, the diffusion 



/(0,t)=0, /(l,t) = 0. 



In this way, the stationary distribution in Eq. fl23|l is not a real final limiting, but a transient 
description of the diffusion process. We term ( 124|) as diffusive landscape. 



B. The validation of KM diffusion approximation in local dynamics 



We now discuss the validation of KM diffusion by comparing the transient landscape if){x) 
and the diffusive landscape 0(x). 

Consider the derivation of 'ip{x), without loss of generality we set w = 1, then 

= 0, 



dip{x) J 
dx 


'T+{xy 


= - In 


{a 
[{c 


-b)x + b 
— d)x + d 


[a — b — c + d) 


is the only solut 


ion 


and 




(x*) = - 


' a — b — c + 
be — ad 


df 


(x*) > 0, then 


near x* 


we have 






ip{x) f 


^ tp{x*) 


+ ^p"{x*] 


(x 


-x*Y 








15 



Meanwhile, 

ld^{x) _ _T+(x) -T-(x) _ _{{a-b-c + d)x+{b~d) _^ 
2 dx ~ ~T+(x) +T-(x) ~ ~ {a-b + c-d)x + {b + d) ~ ' 

its only solution is also x* = {d — b)/{a — b — c + d), and interestingly 

*^ {a-b-c + df n 

From the above comparison, we find that and (\)[x) share the same extremal point 
and the curvature near x* . That is, the Gaussian variance of is equal to that of 
implying that the local movements near the extremal point in the diffusion process are in 
agreement with that in the original Moran process for large populations. 

van Kampen's expansion gives a formal argument to the local validation of diffusion 
approximation. Consider the VK diffusion ( HOl) near x*, then Eq. ( HOl) reduces to a time- 
homogeneous Fokker-Planck equation 

The Gaussian process defined by (|25|1 yields to the following linear stochastic differential 
equation 

d^{t) = -A^{t)dt + DdBt, 

where both A = —-£:{T~^{x) — T~{x))x=x* and D = a/T+(x*) + T~(x*) are constant. This 
process ^{t) is called Ornstein-Uhlenbeck (OU) process j^, whose stationary variance is 
given by 

D'^ be - ad 1 



(25) 



2 A {a-b-c + dy iIj" {x*) 
This is accordance with our result that diffusion approximation gives the same local dynamics 

as the original Moran process for large populations. 

We now realize that not only does KM diffusion theory give the deterministic nonlinear 

dynamical approximation to the Moran process, but it also gives a good approximation to 

the intra-attractoral stochastic dynamics. 



C. The invalidation of KM diffusion approximation in global dynamics 

Until now, it has been shown that the KM diffusion approximation correctly describe two 
kinds of dynamics: 1) Deterministic nonlinear dynamics; 2) local stochastic dynamics. In 
this section, we will further investigate the diffusion approximation for global dynamics. 
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Consider the uni-harrier case when a > c and h < d. In this bistable game system, 
the comparison of different stable strategies is intimately related to the Maxwell-type con- 
struction, which is dependent on the global inter-attractoral dynamics. The Maxwell-type 
construction indicates that except for the critical case, only one strategy should be selected 
as the unique stable one. Therefore, different constructions could lead to different global 
dynamical behavior. 

For il){x), it has been shown that 

6 In 6 — a In a d\nd — c\nc 



For the diffusive landscape however. 



b — a 



d 



0(1) = m 



1 

'0 

2 In 



b — c + d)y + b — d 



a + c 
b + d 



dy = 

{a — b — c + d){a — b + c — d) 



{a-b + c-d)y + b + d 



(a — b)d — (c — d)b 

Fig. 3 shows a simple example: with the payoff parameters that make = ip{0), 

0(1) > 0(0). In this case, the original Moran process and its diffusion approximated process 
will select different transient attractors in large population size. Different global minimum 
searches lead to different strategy selections. Therefore, for the evolutionary game systems 
with multiple stable equilibria, the validity of this diffusion approximation becomes ques- 
tionable in global dynamics. 

In fact, it is not very surprised to see the global dynamical inconsistency between the 
Moran process for large populations and KM diffusion, since their different large deviation 
functions result in different exponential tails of their stationary distributions, which is in- 
timately related to the inter-attractoral dynamics consisting of barrier crossing movements 
from one attractor to another. 

To illustrate this problem, we consider a simple birth-death process Y{t) with birth rate 
fii = fi and death rate Aj = A, i.e. the transition rates are independent of the states js^]. 
Y{t) has a reflecting boundary at M and an absorbing boundary at 0. We are interested 
in Tn the first passage time from n to 45|. In this simple model, there are three kinds of 
movements from n to (let 6 = A/yu): downhill [6 > 1), uphill [6 < 1) and fiat {6 = 1). 

It is not difficult to have that 

1 / 1 - 0" 



A V ^" - 



n 



X — fj, 



(26) 
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Let the space step between i to i + 1 he S , and let 5 — )> and n — oo, but nS — )■ x, then we 
have a Fokker-Planck equation 



df{x,t) ^^d'f 



dx ' 



(27) 



dt dx"^ 

where D = + A)(5^/2 and V = — X)5. The corresponding first passage time for f l27|l is 



D 



1 -X 



(28) 



Now discretizing x as n5, we have 

1 



Tn. 



(/i-A)2(l 



2{l-e)n 

e (1+9) 



n 



X — fi 



(29) 



Comparing r„ and r„ 



hm ^ 

n— >-oo Tr, 



hm — 

n-s>oo (l+e) 
2(1-0) 



More specificaUy, we have 



hm In 

n->oo n T„ 



2(1-9)71 

e (1+9) 



2fl 



00 if 9 <1 

1 if e>i 



(30) 



1 + ^ 



In^, 



^ < 1) 



(31) 



and 



hm n In — 

n— >-oo 7"„ 



(32) 



1 z/ e = 1 

\ife>i 

From the above comparison of r„ and r„, we find that they both approach to n/(A — /i) in 
the downhih dynamics; while in the uphill dynamics, both Tn and r„ share the exponential 
form of ~ e°", but different exponential parameters. This is the heart of our example. We 
should note that, for the bistable systems, the Maxwell- type constructions are determined 
by the jump processes between these two attractors (back and forth), which are both rare 
events with exponentially long time to happen. According to the above disparity between 
r„ and r„ in the uphill dynamics, KM diffusion approximation can not give the exponent 
correctly, and then results in representing the inter-attractoral dynamical inaccurately. We 
suggest this as the reason for the invalidity of the diffusion approximation for the global 
dynamics and landscape. 
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D. Diffusion's dilemma 



According to Kurtz's theorem |5ll . |52| |. KM's diffusion theory can be mathematically 
justified only for any finite time t. In other words, Eq. f lTS]) correctly approximates the 
finite-time Moran process for large but finite populations, whereas it is not guaranteed that 
they share the same long-term stationary behavior. Therefore, the difficulty encountered 
by KM's diffusion in bistable game systems stems from the fact that exchanging the limits 
of population size and time is problematic. It concerns with non-uniform convergence of 
Kurtz's result. 

A natural question is whether one can find a diffusion process that gives both satisfactory 



49| proposed a very 



finite-time and stationary dynamical approximation. Hanggi et al. 
different diffusion process in the context of Chemical Master Equation: 



dt dx'' ^ iV9x2 VlnT+(x) -lnT-(x) 

The heuristic derivation of Eq. fl33p is based on Onsager's theory. Then the stochastic 
potential for the system should be the transient landscape il^{x), and the thermodynamic 
force is 

F{x) = = lnT+(x) - Inr-(x). 

ax 

Therefore, the macroscopic ordinary differential equation should be 

^ = T+{x) - T~{x) = 7]~\x)F{x). (34) 

So 

, _ T+(x)-T-(x) 

lnT+(a:)-lnr-(x)' ^^^^ 
and the diffusion coefficient proportional to ri~^{x). In order to distinguish Hanggi et al.'s 

from KM's, we term Eq. (jSS]) as HGTT's diffusion. 

It is easy to show that Eq. (133|) gives the same large deviation function as the original 

Moran process. Moreover, by comparing the drift coefficients of ([T8|l and (|33|1 . 



aKM^x) = aHGTrix) = T+(x) - T (x), 

HGTT's and KM's describe the same ODE when tends to infinity. For the diffusion 
coefficients: 

h (r) - THx)+T-ix) 
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It is easy to find that HGTT's diffusion coefficient is always smaller than that of KM's (see 
Fig. 4), except bxAiix) ~ bnGTrix) when x near x*. So away from the extremal point, 
HGTT's diffusion shows different finite-time stochastic dynamics from KM's. Note that 
KM's diffusion gives the correct finite-time dynamical approximation of the original Moran 
process, HGTT's could then show a wrong short-term dynamics for most of the initial states. 

Therefore, our diffusion dilemma can be stated as follows: Can we find an approximated 
diffusion process correctly describe the whole three dynamical regimes: (a) The deterministic 
limit; (b) the short time stochastic dynamics; (c) long time global dynamics? For truncated 
KM approximation (and van Kampen's expansion), the (a) and (b) are correct for each and 
very attractor, but (c) is wrong. For HGTT's diffusion, (a) and (c) are correct, but (b) is 
wrong. So we can not find a diffusion process that provides all the three correctly. 



V. DISCUSSIONS 



Stochastic dynamics have 



evolutionary theory 



flfl 



66 



become a fundamental theory in understanding Darwinian 
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Besides nonlinearity^ stochasticity has been shown as 



another basic feature of complexity in biological world 69[, especially within the scale of 
cellular dynamics [zoiJ^]. Stochastic evolutionary game dynamics, as agent-based models 
to describe the kinetics in polymorphic population systems, offer a framework to study the 
frequency- dependent selection in evolution. 

The present paper discuss the well-mixed stochastic evolutionary game dynamics from 
the viewpoint of the transients. The transient landscape, as a potential-like representation 
of the pre-fixation dynamics, has been constructed via the conditional stationary distribu- 
tion in the theory of quasi-stationarity in terms of the large deviation rate function. The 
involvement of large deviation theory from probability is essential here, for without it, the 
landscape would be system's size dependent. It has been shown that this transient land- 
scape can play a central role in connecting the deterministic replicator dynamics, the final 
fixation behavior and diffusion approximation. As a Lyapunov function of the replicator dy- 
namics, the transient landscape visually captures the infinite-population nonlinear behavior. 
The downhill movements in this landscape corresponds to the dynamics of its deterministic 
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counterpart, whereas the rare uphill movements arising from the random fluctuations are of 
more interest in stochastic evolutionary systems. To capture the eventual flxation behavior 
from the transient perspective, we have classified the absorbing states into two cases: The 
attractive absorbing state which is located at the transient attractor; the other rare absorb- 
ing state which is located at the top of the landscape. The former is an inherent result of the 
transient downhill dynamics, while the latter is related to the multiple time scale issue, that 
is, the final fixation time scale is separated from the transient coexistence quasi-stationarity. 

Furthermore, the Maxwell-type construction and diffusion approximation are both impor- 
tant problems linking to the transient dynamics. The Maxwell-type construction is a global 
description of nonlinear bistable stochastic dynamics, which is not present in deterministic 
dynamics. This construction always searches the global minimum in the landscape, so it is 
a direct result of inter-attractoral dynamics. The comparison of the Maxwell-type construc- 
tions between the original transient landscape and its diffusion counterpart indicates that 
the truncated KM diffusion approximation could result in different global dynamics, that is, 
the original Moran process for large populations and its diffusion counterpart could select 
different global stable points. In order to solve this problem, another HGTT's diffusion 
has been constructed for giving the correct long-term asymptotic dynamics. However, this 
diffusion gives the wrong finite time stochastic dynamics. 

By investigating the first passage times in the simple birth-death process, it has been 
found that the failure of exponential approximation in the uphill movement could be a 
reason for our diffusion's problem. Mathematically, the diffusion approximation is just a 
second-order polynomial expansion of the Master equation, which only offers the second- 
order precision for the original process. Accordingly, this approach can give the correct 
deterministic dynamics (first order) and Gaussian dynamics near the stable point (second- 
order). However, the inter-attractoral dynamics is determined by the rare barrier crossing 
movements with exponentially small probabilities, so the Maxwell-type construction should 
be approximated in the level of exponential asymptotics, which could be out of any finite 
order expansions' league. In the theory of probability, this is the domain of the Large 



Deviation Theory 



35|. 



It is believed that discrete stochastic dynamics offers a new perspective on biological 
dynamics. Besides the conventional concentrations on maximum-likelihood events, more 
attention should be paid to rare events. Evolution itself is a process with the accumulations 
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of various rare events, such as genetic or epigenetic mutations and ecological catastrophes. 
So the stochasticity is not just fluctuations near the most probable macroscopic states, but 
an important source of complexity, i.e., "innovation", especially on an evolutionary time 
scale. 

VI. ACKNOWLEDGEMENTS 

We thank Tibor Antal, Ping Ao and Hao Ge for reading the manuscript and helpful 
comments. Discussions with Jiazeng Wang, Bin Wu and Michael Q. Zhang are gratefully 
acknowledged. DZ also wish to acknowledge support by the National Natural Science Foun- 
dation of China (10625101), and the 973 Fund (2006CB805900). 

VII. APPENDIX 

A. Conditional stationary distribution in the theory of quasi-stationarity 

Quasi-stationarity is a series of stochastic mathematical techniques for analyzing the 
Markov processes with absorbing states. The basic idea of the quasi-stationarity is to find 
some effective distributions for characterizing the transient behavior of the process. There 
are basically two kinds of quasi-stationarities: conditional stationary distribution and sta- 



26| for more details. 



tionary conditional one. Here we only consider the former, see 

In order to introduce the conditional stationary distribution, we now add small mutations 
to the original Moran process as follows: 

T+ = P(0 ^l)=e. (36) 

= p{N ^N-l)=e. (37) 

In this case, the process has become irreducible. Further, the stationary distribution of the 
new chain reads: 

i 

fi.{e)=Cl[T^_jT^. (38) 

fc=i 

where C is the normalized constant. Consider 

V^ie) = - ^ ^ = 1,2,...,A^-1, (39) 
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it is not difficult to have that ri(e) is independent of e, and r]{e) is just the same as rj in 



Eq. f fTT]) . It has been shown in [26[ that rjj is proportional to the expected time of visits to 
state j before absorption when started in the revival distribution. That is, rj characterizes 
the occupation time distribution of the transient dynamics. The larger rjj, the longer the 
process stays at state j before absorption. 

Here we should emphasize that, given a Markov chain with absorbing states, the pre- 
fixation occupation time distribution depends on the distribution of the states in which the 
chain is revived. For the birth-death process here, it is natural to choose the reviving states 
as neighboring the absorbing states. 



B. Generalized transient landscape for multi-dimensional cases 



In this section we will show that the definition of the transient landscape in Eq. ( 1121) can 
be extended to more general cases. 

Consider a mult i- dimensional birth-death process with absorbing states, i.e. Xf = 
(Xi(t), X2(t), Xrf(t)). The state space of this process is a dimensional vector space. 



-iV^(f) + ^i(f) + ^ + 



X 



n 

N' 



denoted as N''. In the generalized Moran process [73|, for instance, d is the number of 
strategies, and Xi(t) is the number of individuals with strategy i at time t. 

Suppose Xt has a unique conditional stationary distribution P^{n), where is the 
population size, n G N'^. As a function of A^, Pj^(n) usually has the so called WKB 
expansion js^l for large population size: 

Pj^(n) cx exp 

That is, (f{x) can be obtained from 

^(f)= hm -llnP,^(A^f), 

if the above limit exists. We define (p{x) as the generalized transient landscape. 

It has been shown that ip{x) still has the Lyapunov property with respect to its ther- 
modynamic limit jlJ]. Suppose the thermodynamic limit of Xt can be described as the 
following deterministic differential equations 



^ = d{xt), Xt = Xt/N. 
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In particular, for the generalized Moran process 73| . 



a[x) = [...,ai[x), ...J 



where Tij (x) is the frequency-dependent probability that an i strategist is replaced by a j 



strategist. From [37|, 



7411. we have 



dx 

ipixt) = \7^{x) ■ 

= \7ip{x) ■ a{x) 

< 0. 



C. van Kampen's expansion 

van Kampen's expansion provides another systematic method of diffusion approximation 



501 1 . The idea of VK expansion is that, in large population size N, the number we are 
interested in (e.g. the number of strategy A) is expected to consist of two parts: deterministic 
and fluctuations parts. Consider the continuous time birth-death process here (the discrete 
time case is similar), for any state i, we have 

i = Nx{t) + N^/^^{t), 

where x{t) is of order A^~^, ^(t) is of N~^^'^. Define the shift operators as uj{Ti) = Tj+i and 
u~^(Ti) = Tj_i, so the Master equation can be written as 

^ = (C.-1 - l){TtPm + {uj- l)(TrP,(z)), 

where Tj"*" is the birth rate, and Tj~ is the death rate. Now we denote the distribution of 
^{t) as n(^,t). In fact, 

U{U) = Pt{Nx{t)+N'/^at)), 

and we have 

dPS) ^ dUjU) ^ dU{^,t) d^ ^ dUjU) _ j,n/2 dm,t) dxjt) 
dt dt d^ dt dt d^ dt 



We take the Taylor expansions: 



uj-l^N-"^— + -N-^ — 
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2 d^' 



dx 
where 

T+ix) = lim ^,T-(x) = lim 

So 



dt di dt \ 2 



The terms of order N^^"^ on either side will vanish if x{t) satisfies the equation 

^ = T+(x)-T-(x), 

which is just the deterministic replicator dynamics. If consider the terms of order A^°, ^(t) 
should obeys 

This is a linear Fokker-Planck equation whose coefficients only depend on x{t). So van 
Kampen's approach gives the correct dynamics conditioned on the deterministic solution. If 
we substitute z = N~'^^'^^ + x{t), we can find that Eq. fHOj) is exactly the same as Eq. f|T8l) . 



D. The relations between transient landscape and fixation probability 

Eq. f|T6l) shows that the fixation probabilities, pj = 1 — 7-,- and our transient landscape 
iplx) have the following relation: 

lim ^ In ^pnx = lim ^ In ^-fN{i-x) = ij{x). (41) 

Af->oo A* dx N^oo IS dx 

To further illustrate this, let us consider a similar relation in a diffusion process with the 
following stochastic differential equation 

dx = a{x)dt + -^h{x)dBt (42) 
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with absorbing boundary conditions. As shown in Sec. IV, the conditional stationary 
distribution can be obtained by solving the Kolmogorov forward equation 



^ {a{x)7r{x)) + {b'{x)n{x)) = 0, 



(43) 



where the stationary distribution is 



7r(x) oc exp(— A^0(x)), 



(44) 



and the transient landscape is 



(j){x) 



/•X 


' a{y) ' 


/ ' 

Jo 





dy. 



(45) 



On the other hand, the fixation probability from a; to 1 is the solution of the backward 
equation [45 1 

(46) 



with boundary conditions 



It is not difficult to show that 



so we also have 



p(0) = 0, p(l) = l. 



p{x 



lim — In — nix) = (hix). 
N^ocN dx 



(47) 



We now attempt to generalize the above relation in Eq. ( 147|) to the more general multi- 
dimensional cases. Consider an n- dimensional diffusion process with forward equation 



1 " r) 1 



X e D, 



(48) 



where the absorbing boundary of D is denoted as dD. For any a G dD, the fixation 
probability density at a from x also satisfies the backward equation 

d 1 A „ a 9 



a 1 
^ ^*(^)^^-(«) + ^ 5Z ^^^^ 



dxi dxj 



pg{d) = 0. 



(49) 



Its boundary condition is 
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where is the Dirac-delta function for dD. 

The conditional stationary distribution solves the {■ ■ ■ } = — Ji 
V ■ J{x) = 0. Detailed balance, however, further dictates Ji{x) = 



x) in Eq. fHHl) . where 



45|. Therefore, 



1 " 



Ai(x) 



1 

2ivE 



(50) 



where is our transient landscape. 

Now consider Eq. fHOl) in the light of fl50l) . First we denote = V^p^(a). It satisfies 



n n 



1 ^ r 1 «9 

9 2^ 



9 1 1 " d 

Bij{x) - Bij{x)—^{x) Ci{x) + ■7^^Bij{x)—Q{x) 



dxj 



1 " 

4e 



n 



(X 



1 " (9 

7 -1 



(51) 



We see a hint of Eq. P7|) in the square bracket. For multi-dimensional problems, the gradient 
of ps{a) is a vector while 0(x) is a scalar. Therefore, it seems to us, even with detailed balance 
condition, the relation in Eq. ( l47l) can not be generalized to mult i- dimensional case. On the 
other hand, the definition of </)(x) can be generalized to mult i- dimensional case (see appendix 
B), even though finding it will be hard. 



[1] M. A. Nowak, Evolutionary Dynamics (Harvard University Press, Cambridge, MA, 2006). 
[2] A. Traulsen and C. Hauert, in Reviews of Nonlinear Dynamics and Complexity, edited by H. 

G. Schuster (Wiley- VCH, Weinheim, 2009). 
[3] C. P. Roca, J. A. Cuesta and A. Sanchez, Phys. Life Rev. 6, 208 (2009). 
[4] R. Durrett, Probability: Theory and Examples (Duxbury Press, Belmont, 1996), second ed. 
[5] S. Karlin and H. M. A. Taylor, A First Course in Stochastic Processes (Academic, London, 

1975), second ed. 
[6] R. A. Fisher, Proc. R. Soc. Edinburgh. 42, 321 (1922). 
[7] S. Wright, Genetics. 16, 97 (1931). 

27 



[8] P. A. Moran, The Statistical Processes of Evolutionary Theory (Clarendon, Oxford, 1962). 

[9] M. A. Nowak, A. Sasaki, C. Taylor, and D. Fudenberg, Nature (London) 428, 646 (2004). 
[10] A. Traulsen, J. M. Pacheco and M. A. Nowak, J. Theor. Biol. 246, 522 (2007). 
[11] F. Fu, M. A. Nowak and C. Hauert, J. Theor. Biol. 266, 358 (2010). 
[12] C. Taylor, D. Fudenberg, A. Sasaki and M. A. Nowak, Bull. Math. Biol. 66, 1621 (2004). 
[13] T. Antal and I. Scheuring, BuU. Math. Biol. 68, 1923 (2006). 
[14] C. Taylor, Y. Iwasa, and M. A. Nowak, J. Theor. Biol. 243, 245 (2006). 
[15] H. Ohtsuki, P. Bordalo, and M. A. Nowak, J. Theor. Biol. 249, 289 (2007). 
[16] P. M. Altrock and A. Traulsen, New J. Phys. 11, 013012 (2009). 
[17] B. Wu, P. M. Altrock, L. Wang and A. Traulsen, Phys. Rev. E. 82, 046106 (2010). 
[18] D. Fudenberg, M. A. Nowak, C. Taylor and L. A. Imhof, Theor. Popul. Biol. 70, 352 (2006). 
[19] T. Antal, M. A. Nowak and A. Traulsen, J. Theor. Biol. 257, 340 (2009). 
[20] T. Antal, A. Traulsen, H. Ohtsuki, C. E. Tarnita and M. A. Nowak, J. Theor. Biol. 258, 614 
(2009). 

[21] G. Szabo and G. Fath, Phys. Rep. 446, 97 (2007). 

[22] M. Droz, J. Szwabihski, and G. Szabo, Eur. Phys. J. B 71, 579 (2009). 

[23] G. Szabo, A. Szolnoki, M. Varga, and L. Hanusovszky, Phys. Rev. E 82, 026110 (2010). 

[24] J. Huisman and F. J. Weissing, Nature (London) 402, 407 (1999). 

[25] J. Huisman and F. J. Weissing, Ecology 82, 2682 (2001). 

[26] J. N. Darroch and E. Seneta, J. Appl. Prob. 2, 88 (1965). 

[27] A. Hastings, Trend Ecol. Evol. 19, 39 (2004). 

[28] G. Block and L. J. S. Allen, Buh. Math. Biol. 62, 199 (2000). 

[29] J. C. Claussen and A. Traulsen, Phys. Rev. E 71, 025101(R) (2005). 

[30] Y. Tao and R. Cressman, Buh. Math. Biol. 69, 1377 (2007). 

[31] M. Vellela and H. Qian, Bull. Math. Biol. 69, 1727 (2007). 

[32] S. G. Ficici and J. B. Pollack, J. Theor. Biol. 247, 426 (2007). 

[33] C. M. Grinstead and J. L. Snell, Introduction to probability (American Mathematical Society, 

Providence, 1997), 2nd edition. 
[34] M. L Freidlin and A. D. Wentzell, Random Perturbations of Dynamical Systems (Springer, 

New York, 1984). 

[35] A. Dembo and O. Zeitouni, Large Deviations Techniques and Applications (Springer- Verlag, 

28 



New York, 1998). 
[36] H. Touchette, Phys. Rep. 478, 1 (2010). 
[37] H. Qian, Nonlinearily. 24, 19 (2011). 

[38] J. Wang, L. Xu and E. K. Wang, Proc. Natl. Acad. Sci. 105, 12271 (2008). 
[39] P. D. Taylor and L. Jonker, Math. Biosci. 40, 145 (1978). 
[40] D. Zhou, B. Wu, H. Ge, J. Theor. Biol. 264, 874 (2010). 

[41] J. Maynard Smith, Evolution and the Theory of Games (Cambridge University Press, Cam- 
bridge, 1982). 

[42] H. Ge and H. Qian, Phys. Rev. Lett. 103, 148103 (2009). 

[43] R. G. Strongin and Y. D. Sergeyev, Global Optimization with Non-Convex Constraints: Se- 
quential and Parallel Algorithms (Kluwer Academic Publishers, Dordrecht, 2000). 
[44] W. Feller, Trans. Amer. Math. Soc. 77, 1 (1954). 

[45] C. W. Gardiner, A Handbook of Stochastic Methods: for Physics, Chemistry and the Natural 
Sciences (Springer, Berlin, 1983). 

[46] W. Y. Tan, Stochastic Modeling of AIDS Epidemiology and HIV Pathogenesis (World Scientific 
Pub, Singapore, 2000). 

[47] A. Traulsen, J. C. Claussen, and C. Hauert, Phys. Rev. Lett. 95, 238701 (2005). 

[48] F. A. C. C. Chalub and M. O. Souza, Math. Comput. Modeh. 47, 743 (2008). 

[49] P. Hanggi, H. Grabert, P. Talkner and H. Thomas, Phys. Rev. A 29, 371 (1984). 

[50] N. G. van Kampen, Stochastic Processes in Physics and Chemistry (Elsevier Science, North- 
Holland, 1992). 

[51] T. G. Kurtz, Math. Prog. Stud. 5, 67 (1976). 

[52] T. G. Kurtz, Stoch. Proc. Appl. 6, 223 (1978). 

[53] H. Qian, Prot. Sci. 11, 1 (2002). 

[54] I. Naseh, Math. Biosci. 156, 21 (1999). 

[55] H. Ge and H. Qian, J. Roy. Soc. Interface. 8, 107 (2011). 

[56] J. M. T. Thompson and H. B. Stewart, Nonlinear Dynamics and Chaos (John Wiley, New 
York, 1986). 

[57] H. K. Khalil, Nonlinear Systems (Prentice Hall, Upper Saddle River NJ, 2002). 
[58] H. Eyring, J. Chem. Phys. 3, 107 (1935). 
[59] H. A. Kramers, Physica. 7, 284 (1949). 

29 



[60] D. Y. Chen, J. F. Feng and M. P. Qian, Science in China (Series A), 39, 7 (1996). 

[61] M. Vehela and H. Qian, J. R. Soc. Interf. 6, 925 (2009). 

[62] H. Qian, P. Z. Shi and J. Xing, Phys. Chem. Chem. Phys. 11, 4861 (2009). 

[63] M. Kimura, Cold Spring Harbour Symp. Quant. Biol. 20, 33 (1955). 

[64] A. J. McKane, D. Waxman, J. Theor. Biol. 247, 849 (2007). 

[65] D. Waxman, J. Theor. Biol. 269, 79 (2011). 

[66] W. H. Sandholm, Population Games and Evolutionary Dynamics (MIT press, Cambridge, 

MA, 2011). 
[67] P. Ao, Phys. Life Rev. 2, 117 (2005). 
[68] P. Ao, Commun. Theor. Phys. 49, 1073 (2008). 

[69] L. J. S. Allen, An Introduction to Stochastic Processes with Applications to Biology (Prentice 

Hah, Upper Saddle River, NJ, 2003). 
[70] M. B. Elowitz, A. J. Levine, E. D. Siggia and P. S. Swain, Science 297, 1183 (2002). 
[71] L. Cai, N. Friedman and X. S. Xie, Nature (London) 440, 358 (2006). 

[72] D. A. Beard and H. Qian, Chemical Biophysics: Quantitative Analysis of Cellular Systems 

(Cambridge University Press, Cambridge, 2008). 
[73] A. Traulsen, J. C. Claussen and C. Hauert, Phys. Rev. E 74, 011901 (2006). 
[74] G. Hu, Zeit. Phys. B, 65, 103 (1986). 



30 



Captions 



Figure 1 (Color online): Transient landscapes and conditional stationary distributions: 
(a) Uni-well case: The small window shows the transient landscape ip{x)] The large window 
shows the conditional stationary distribution ri{x) with different population size. Parameters 
are a = 1, b = 2, c = 3, d = 1, and w = 0.7. (b) Uni-barrier case, with parameters a = 2.5, 
b=l,c=l,d = 2 and w = 0.7. (c) Uphill case, with parameters a = 1, b = 1.2, c = 1.5, 
d= 1.4. 

Figure 2 (Color online): Maxwell- type construction for the bistable Moran process: (a) 
When the critical condition is satisfied {a = 2, b = 1, c = 1, d = 2), ip{0) = Both are 

equally important, (b) With parameters a = 2.5, b = 1, c = 1, d = 2, tp{0) > Then 
77(0) < 77(1), and even 77(0) ^ 77(1) for large population size. 

Figure 3 (Color online): The original Moran process and its KM approximated diffusion 
process show different Maxwell-type constructions. In this example, = ip{0), but 

0(1) > 0(0). (The figure is magnified and we focus on the region near x = 1) 

Figure 4 (Color online): The HGTT's diffusion coefficient is always smaller than KM's, 
except at x* where they are both equal to each other. 
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